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Abstract —In this paper, we propose a novel formulation 
to understand the tradeoff between binning utility and meta 
information leakage when we face the problems of partitioning 
random items. As an example, such problems could emerge 
when online users attempt to protect their browsing behavior 
patterns to certain extent by resorting to multiple proxy 
websites. Under the framework, we formulate a constrained 
optimization problem where the goal is to maximize binning 
utility while restraining a certain level of information leakage 
by properly dividing a set of M random items into N = 2 bins. 
We then offer algorithms and complexity orders to solve such 
a problem. These developments will be the cornerstone for our 
further investigation of more generalized scenarios. 

Index Terms —partition, information leakage, privacy, mutual 
information, submodularity 

I. Introduction 

Today internet has become so intertwined with our every¬ 
day activities that it is impossible to imagine living without 
it. However, this dependency could be subject to exploitation 
by the eavesdroppers. Assuming every browse as a query, it 
could be argued that the search history of a user contains 
a series of queries specific to him which could point to his 
specific likes and dislikes. By following every user’s history 
of browses vital information specific to each user could be 
developed. It then becomes important that each user attempt 
to add some level of protection to their browses to hide 
necessary -or rather enough information about themselves. 

To do so, the user has some options. First option is for him 
to use proxy softwares also known as VPNs; such programs 
fully change the user’s IP address rendering him undetectable 
and thus futile for the service provider to track. The problem 
with such softwares is the fact that they slow down the 
connection -such as the incognito mode in different browsers. 
Thus, while all the information does stay hidden, a reduction 
of speed prevents this method from being highly desirable. 

Another method is the use of proxy websites. Such web¬ 
sites offer the user a URL box where he can input any 
website he wishes to visit. The only difference is that in 
such websites, the address input is decoded into a series 
of characters which appear at the end of the URL of the 
original proxy website. These characters change through time 
by seconds meaning if the service provider records the URL 
opened through the proxy website and decides to open it 
to access specific content by inputting the URL, he will 
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not go to the decoded web page. Still, such websites slow 
the connection. However, through this method, the user has 
the option of choosing multiple proxy websites and thus 
presumably cutting down on the utility loss. Thus if a utility 
function based upon connection speed -bandwidth- for the 
user is calculable, a privacy constrained problem between 
the user and an eavesdropper (for example a service provider) 
could be defined. The solution to such a problem could offer 
insights in regards to the tradeoff between proxy allocation 
utility and meta information leakage when we face the 
problems of partitioning a set of random items (i.e. websites 
that a user has chosen to visit following his own distributions) 
into a given number of bins (i.e. a given set of proxy servers 
each of which has its own utility function, as will be further 
detailed in Section II). 

In our proposed framework as detailed in Section II, meta 
data information refers to the patterns about a sequence of 
itemsffor example users favorable websites) infer-able based 
upon a sequence of bins (e.g. proxy sites) observed by an 
eavesdropper. This assumption is an expansion of [1] and [2], 
However, it should be noted that due to usage of proxy sites, 
an eavesdropper cannot directly observe the original and 
input items, but rather bin indexes. Under our proposed novel 
framework, we further propose two approaches, based upon 
exhaustive search and submodular functions , respectively, 
to solving the formulated constraint optimization problem, 
namely, dividing M random items into N = 2 bins, under 
an upper-bound on leaked meta information. 

This paper represents an expansion of our previous works 
in [3], [4] where in [3] a utility function was introduced 
in the form of the average stopping time for detection of 
an active subgraph using certain queries while in [4], we 
developed a concept of information leakage through a vast set 
of possible queries. In this paper, we shift our attention from 
detection of an active subgraph to seeking tradeoff between 
optimizing utility of partitioning a set of random items 
and restraining information leakage. From the perspective of 
[3,4], a set of random items could represent a set of edges 
in a given subgraph, which become active probabilistic-ally, 
while a partition strategy corresponds to a particular option 
of querying edges. 

The rest of the paper is organized as follows. In Section 
II we formulate the problem in terms of privacy and utility 
functions. We dissect what the goal and the constraints 
are. We then explain why due to imposed complexities we 
have chosen to simplify the problem to only 2 bins in 



Section ??. In Section ??, we offer a full algorithm to an 
exhaustive search solution to a specific expansion of our 
problem; multiple algorithms are further developed and their 
corresponding complexities are addressed. In Section III, we 
show the conditions under which both the objective and 
constraint functions are equipped with submodular property 
under which we can leverage efficient algorithms to seek the 
optimal partition of random items for more general scenarios. 
Finally, conclusions are drawn in Section ??. 

II. System Model 

First, we propose an abstract framework to formalize the 
goal of seeking partition of TV items into M bins. More 
specifically, we aim to allocate each one of 1 < i < M 
possible items (queries) to one of TV output bins. There 
could be at most N M such partitions. It follows that any 
set allocation Ai, 1 < l < N M results in TV sets S^ C 
{1,2,..., M},j = 1, 2,..., TV. Each such set is debited as 

sf = = i} 

where 9 E (1) 

3 |0 i$ S?> 

We further assume S'® D S^ = 0, j ^ k. Furthermore we 
have UjLi Sj 1 = {1, 2,..., M}. Finally the size of each such 
set Sj 1 ' 1 is debned as L ( 'p. 

A. Probabilistic Model 

We assume at any time slot one and only one of the inputs 
is chosen with a certain probability. Thus if we use variable 
A' £ {1,2as a representation of set of items, we 
could have P(X = i) = P(p/i = 1) = 7 r*, 1 < i < M 
as a representation of the probability of choosing item i 
from the set X where 7 j £ {0,1}. It further follows that 
TldLi 7* = 1- stipulating that one and only one of M items 
is selected. These M items could represent a set of M web 
pages to be visited by a user at a particular time instant. The 
prior probability distribution of M items rebects the user’s 
favoritism toward these web pages. . 

Next, we introduce an observable random variable 
Y £ {1,2,•••,7V}, denoting the index of the bin (the 
proxy site) employed to carry one of the M > N items. 
It follows that the probability of each bin’s appearance 
given a set allocation scheme such as Ai will be equal to 

P(Y = j\A t ) = P(Y = Mu X = i)P{X = i\ A t ) = 
HiLi P(Y = j\Ai, X = i)P(X = i) where we have 
dropped the second conditional probability due to 
the independence between X and Ai. Furthermore 
P(Y = j\A/.X = i) = 9j l j £ {0,, 1}. It thus follows that 

M 

P{Y=j\A l ) = Y J 0^P{X=i)^ 

i—1 

P(Y = Ml) = E ^ = a f ( 2 ) 


B. Revealed Information 

By choosing to allocate M items to TV bins where TV < M, 
we have injected ambiguity and uncertainty into the output 
binning index sequence about the input item sequence over 
a successive n visits or channel uses. In other words, if we 
originally chose to transmit n of suc h it ems, our total set of 
possible sequences would be of form X n = [A' | X 2 ...X,,] out 
of M n possible outcomes. From an observer’s perspective 
which can only have access to which one of TV bins is 
deployed in each time slot, sequences in the form of Y 1 ' 
= [YiY 2 ...Y n ] has cardinality of at most N n < M n . Despite 
the amount of uncertainty added due to the many-to-one 
mapping between items and bins, the output sequence sill 
reveals certain amount of information regarding the patterns 
of sequences of M random items. 

This observation could be further studied by indicating 
how our allocation system resembles a coding framework 
where we have an equivalent channel whose input variable 
is X and output Y, as show in Figure 1. 



Figure 1. Coding Channel Representation of the Problem 

Under such a framework, the equivalent channel output 
sequence Y can help an eavesdropper classify the input 
sequence X into a number of differential classes. As a 
result, information about the specific input item patterns 
is leaked to certain degree and can be measured using 
conditional mutual information I(X;Y\Ai) between X and 
Y, given a particular channel mapping (i.e. partition Ai 
relationship as illustrated in Figure 1. 

Such conditional mutual information thus measures the 
maximum number of bits of meta information about item 
sequence per channel use. Therefore, we can have at most 
2 ni(X;Y\Ai) se q Uences X" distinguishable by inferring based 
on Y". We thus adopt I{X\Y\A{) as the privacy metric 
conditioned on a particular partition mapping Ai. 

It follows that due to the combinatorial nature of a set 
allocation problem there are a total of N M possible methods 
to allocate these M items to the TV sets. We can formulate 
the mutual information over a set allocation Ai : 1 <1<N m 
as: 

I(X-,Y\At) = H(X\A t ) - H(X\Y,A[) = 

H(Y\Ai) - H(Y\X , A,) = H(Y\A[) = 

..,«$) (3) 

where we have used the notion of H(ai, <3,2, ..., a m ) = 
— X)"Li a v log a v and the fact that H(Y |A, Ai) = 0 because 




if both the input X and the channel scheme Ai are known, 
then output Y could simply be calculated. 


C. Utility Function 

Note that the main reason why we chose bin allocation 
was to reach a higher utility function. In this section we 
define a utility function to apply to the problem. If an 
allocation scheme Ai has resulted in S'®, S®, ..., S®, we 
assume each of the N bins offer a utility function of their 
own based upon the set they have been bestowed. Every 
bin j thus offers a utility represented by / 7 (S®). It then 
follows that the overall utility function will be in the form 

of Ut = Ef=i fj(sf)P(Y = j\A t ) = Ef =1 afhisf). 


D. Problem Definition 

Based on the previous observations we set a goal to find 
the set allocation Ai over which (a) Ui is maximized and (b) 
I(X;Y\A[) < I t h where I t f t represents the maximal allowed 
revealed information. 

We aim to find the set allocation scheme Ai which helps 


max 
1 <i<n m 


N 


^ = £ 




3=1 


while fT(a®, a® > —, 0 $) < hh 


and 


M _ 


= £ 


ies 


a) 


(4) 


As shown in Eq. 4, we in general have to deal with an 
exponentially large number N M of candidates, under a given 
N and with respect to M. 


III. Multi-Submodular Set Functions as a Means 
of Solution 

As mentioned previously, the problem formulated in Eq. 
(4) is NP-complicated. Still, we could opt to utilize the 
definition of multi-submodular set functions so as to reduce 
the complexity to that of polynomial at the cost of error. In 
order to do so, we first introduce a new realization of the 
problem. We then express concerns about possible solutions. 
Next, we offer insight as to how we could deal with each 
case. 


A. New Problem Development 

We aim to gather both the utility and the constraint 
imposed in Eq. (4) in the form of one function we hope 
to maximize. Thus, a new maximization problem could be 
developed: 

max Ui + \(I th - H(a^ ) , a$)) (5) 

l<t<JV M 

where A > 0 represents a variable connecting the utility 
and the constraint to each other so as to allow comparison 


between them. We can further express this equation by 
opening it: 

N 

£[af/i(Sf) + Aa®!og(a®)] (6) 

3 = 1 

We can now see that if we define F® = a®/,(S®) + 
Act® log (a®), Equation (6) is simply a sum of functions 
defined over a series of sets. We refer to these as multi¬ 
variate set functions seeing as how their values are based 
upon specific sets and variables introduced in each of these 
sets. 

B. Imposing Multi-submodularity 

[5] first proved that if we can prove multi-submodularity 
for functions such as those formulated in Eq. (6), then 
they could be modeled as simpler problems (submodular set 
functions). We thus, aim to find the sufficient conditions for 
such occurrence. To do so, we first offer a definition of multi¬ 
submodularity. 

As mentioned in [5], a multivariate function F : N M —> R 
is multi-submodular if for all pairs of tuples (Si, ..., Sm) and 
(Ti,... Tm) € N M we will have: 

F(Si ,..., S M ) + F(Ti, ..., T m ) > F(S 1 U T u ..., S M U T M ) 
+F(5 1 nT 1 ,..,s M nr M ) (7) 

Since in our formulation functions are separately defined on 
different sets, the condition in Eq. (7) is simplified to the 
sufficient condition of submodularity of F® for all sets S t 
for Equation (6). We now need to find the sufficient condition 
for submodularity of F® when defined over a set Sj. 

C. Imposing Separate Submodularities 

For an easier mathematical representation of the follow¬ 
ing derivation we assume F(Sj) = F®. Without loss of 
generality we also assume that /y (S®) = /(S'®). The first 
assumption simply dictates that once a mapping scheme Ai 
is chosen, its index does not play any further role in our 
proof. The second assumption necessitates the utility function 
defined over all sets to be the same. 

In the next step, we opt to use diminishing return property 
as the means of making certain each of these functions are 
submodular. Following is a definition of diminishing returns 
for submodular functions, after which we derive the sufficient 
conditions for the case discussed in Eq. (6). 

Diminishing Property Return dictates that a set function 
F : 2 n —* R is submodular if, for all A,BCtt with A C B 
and for each x £ Q — B we have [6]: 

F(AU{x})-F(A)>F(BA{x})-F(B) (8) 

Now we attempt to expand Eq. (8) for each F(Sj). However, 
to properly do so, we first need to account for the behavior 
of this function. 

We have defined F(Sj) = ctjf(Sj) + Xoij log (ay ) where 
it seems that the function has a singular relationship with set 



Sj. However; there is a secondary relationship the function 
shares with the set Sj = S — Sj where S represents 
the universal set. This relationship could be modeled as 
F(Sf) = (1 — Pj) f (S — S^) + X(1 — Pj) log (1 — pj) where 
we have used the fact that Pj = 1 — atj seeing as how we 
define 

Pf = E ^ (9 > 

i£{S-S) l> } 

Thus, for any set Sj, we must find the sufficient conditions 
for the existence of diminishing property for both functions 
(Sj) = Ujf(Sj) + Xaj log (aj) and F 2 (Sj) = (1 - 
aj)f(S — Sj) + A(1 — aj) log (1 — aj). To do so, we will 
calculate their necessary conditions and then find their (T 
as the final conditions (assuming they do not negate one 
another). 

Note: For any further references, we first need to address 
a series of variable and function definitions which are going 
to play a vital role in the rest of this paper: 

Definitions 

1. Any variable represented with a capital Letter represents 
a set. 

2. Any variable represented with a small letter represents 
an element. 

3. a x represents the probability of item x and cxa repre¬ 
sents the sum of probabilities of items mapped into a 
set A. 

4. olba represents the difference in the sum of probabili¬ 
ties of items mapped into the sets B and A which could 
be further shown as ciba = oib — a a. 

5. g(C) represents the 1 st order difference of a set 
function /(c) which could be formulated as g(C) = 

f(C) ~ f(C — D). 

6. q(C) represents the 2 nd order difference of a set 
function /(c) which could be formulated as q(C) = 

g(c) — g{C — D). 

7. We assume the probability of items is sorted in a 
decreasing manner as 7ri > it 2 > ... > tvm. 


Lemma III.l. The function F-\ (Sj) is submodular if 

( 1 ) g(C) <0 

(2) q(C) < 0 

(3) \g(C)\ > Alog(^) 

for all sets C. 

The proof for this lemma is presented in Appendix. 

Lemma III.2. The function F 2 (Sj) is submodular if 

( 1 ) g(C) < 0 

(2) q{C) < 0 

(3) \g(C)\ > A log (^) 
for all sets C. 

The proof for this lemma is presented in Appendix. We 
can then use the results from two lemmas above to conclude 
that: 

Corollary III.2.1. The function F{Sj) is submodular if 

( 1 ) g(C) < 0 


(2) q{C) < 0 

(3) | 5 ((7)|>Alog(£) 
for all sets C. 

Unfortunately, [5] does not provide us with an algorithm 
to remodel our multi-submodular problem in a submodular 
problem, they simply prove that this could be done. Thus, in 
order to expand upon the idea of polynomial complexity of 
solution algorithms we opt to assume N = 2 and offer the 
reader the algorithm to deal with such a specific case. We 
then calculate the complexity imposed by the algorithm to 
further stress the benefits of using such an idea in spite of 
accepting error. 

Lemma III.3. When N = 2, the function in Eq. (5) is 
submodular if 

(1) g{C) < 0 

(2) 2\g{C)\ > Alog (K), K < (t^-) 2 
for all sets C. 
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Appendix 

A. Lemma III.l 

Proof. Starting for F-\ (Sj), by rewriting Eq. (8) we will have: 

(a a + a x )f(A + {x}) + A (a a + ot x ) log (a a + a x ) 

-a A f(A) - XaA log (a a) > 

(a a + as a + ot x )f{B + {x}) 

+A(«a + otBA + oc x ) log (aA + asA + ot x ) 

~{a A + a B A)f{B) - X(a A + qm) log {a a + Wba) 

We can then factorize the inequality as 

0iBA[-f{B + {x}) + f(B) - A log (aA + a B A + a x ) 

+A log (a a + a B A)] + a x [f (A + {x}) 

-f(B + {x}) + Alog (a a + a x ) - Alog (a a + oi B a + oi x )] 
+a A [f(A + {x}) - f{B + {x}) - f(A) + f(B) 

+A log (a A + a x ) - X log (a a + a BA + a x ) 

-A log (a a) + X log (a a + a BA )} > 0 




B. Lemma III.2 


We assume g(x + y) = /( x + y) - /(x), q{x + y) = g(x + 
y) — g(x),y > 0. We can then rewrite the factorization as: 

a BA [~g{B + {x}) + A log [ - 1 Qj ]] 
+a x [-g(B + {x}) + A log ^ \ nA ]] 

CKA+CKx 
^ _|_ &BA 

+a A [~q{B + {xx}) + A log - > 0 

' a A +a x 

Since the above inequality needs to hold true for all possible 
sets of A C B, x B. we aim to determine the maximal 
amount enforced by the above set of inequalities. The first 
factorization results in two inequalities: (1) g{C) < 0 for 
all sets C and (2) \g{C)\ > Amax(log(l + ^j)). To find 
the maximum of such a limit, we need to impose the one 
item with highest probability to {x} and assume the one 
lowest probability item to set B. Then the above inequality 
is maximized. 

It is important to note that by doing so, we are removing 
the possibility of B = 0 as a case. In other words, we 
are assuming that a B = 0 is not a scenario we need to 
discuss. We will now demonstrate why such an assumption 
is accurate. 

When B = 0, we can conclude that a A = at B = 0 and 
then rewrite inequality (8) in the following format: 

> & x f({x}) 

which is always true. 

The second factorization results in two inequalities: 
(1) g{C) < 0 for all sets C and (2) \g(C)\ > 

A?nax(log (1 + 4+a ))• To find the maximum of such a 
limit, we need to impose the item with lowest probability to 
{x} and that a a = 0 and then have a BA = 1 — ot x = a B . 

Once again, it is important to note that by doing so, we 
are removing the possibility of a A = 0 and a x = 0 as a 
case. In other words, we are assuming that a a + a x = 0 is 
not a scenario we need to discuss. We will now demonstrate 
why such an assumption is accurate. 

When a a + a x = 0, we can rewrite inequality (8) in the 
following format: 

0 > a BA f(B) - a BA f{B) = 0 

which is always true. 

The third factorization could be simplified. The logarithm 
argument consists of a nominator greater than denomina¬ 
tor thus resulting in the overall logarithm argument to be 
positive. We thus only need to impose that q(C) < 0. It 
then follows that the probability of each item is sorted in 
a decreasing manner as 7 Ti,...,7Tm> we can write the set 
of 3 sufficient conditions for submodularity of set function 
Fi(Sj),j = 1, N as (1) g(C) < 0 (2) q(C) < 0 (3) 
\g(C)\ > A log [max(l + %,1 + ^)] = Alog(^). 

□ 


Proof. We now follow the same method for F-ziSjf.j = 
1,..., N by rewriting Eq. (8): 

(1 - a A - a x )f(S -A- {x})+ 

A(1 -a A - a x ) log (1 - a A - a x ) - (1 - a A )f(S - A) 
—A(1 - a A ) log (1 - a A ) > 

{l-a A ~ a BA - a x )f(S - B - {x}) 

+A(1 — a A — a BA - a x ) log (1 — a A - a BA - a x ) 

— (1 — a A — a BA )f{S — B )— 

A(1 -a A - a BA ) log (l-a A - a BA ) 


We can then factorize the inequality as 

a BA [f(S - B - {x}) - f(S - B) 

—A log (1 + --^-)] 

1 — OLA — &BA — &x 

+u x [-f(S -A- {x}) + f(S - B - {x}) 

—A log (1 + ---)] 

1 — a a — a x — a B A 

+a A [-f(S - A~ {x}) + f(S - B - {x}) + f(S - A) 
r-fc n\ i \ l ( 1 - a A 1 - Oi A - a BA - a x 

-f(S - B) + A log (-- - -)] 

l — a A — a x 1 - a A - a BA 

+1 lf(S -A- {x}) - f(S-B- {x}) - f(S - A) 
+f(S — B) + A log A aA ° x .. 


1 - OL A 
> 0 


1 — a A — a BA — a x 


We assume g(x + y) = f(x + y) - /(x), q(x + y) = g{x + 
y) — g(x),y > 0. We can then rewrite the factorization as: 


a BA [-g(S - B) - A log (1 + 


1 — a A — a BA — a x 

+a x [-g{S - A - {x}) - A log (1 + - —— -)] 

1 — a A — a x — a BA 

. r to /n , m t 1 ~ a A 1 - a A - a BA - a x v 
+a A \q{S - A) + Alog(: 


+l[-g(S - A) + A log ( 


1- a A -a x 
1 - ola - ol x 


1 — OLA — a BA 

1 — OLA ~ OL B A 


1 - ol a 1 — ol a - ol ba — a. 

> 0 


-)] 


Since the above inequality needs to hold true for all possible 
sets of A C B, x f. B, we aim to determine the maximal 
amount enforced by the above set of inequalities. The first 
factorization results in two inequalities: (1) g(C) < 0 for 
all sets C and (2) \g(C)\ > Alog(l + )• To find 

the maximum of such a limit, we need to impose the one 
item with highest probability to {x} and assume that as is 
maximal while still less than 1 — a x . This limit is imposed 
so that the denominator is not equal to 0. 

Once again, it is important to note that by doing so, we 
are removing the possibility of B = {x} c as a case. In other 
words, we are assuming that a B + a x = 1 is not a scenario 
we need to discuss. We will now demonstrate why such an 
assumption is accurate. 




When olb + a x = 1, we can rewrite inequality (8) in the The third and forth factorizations could be simplified. We 
following format: could write them as 


(1 - a A ~ a x )f(S -A - {x})+ 

A(1 - a A - a x ) log (1 - a A - a x ) - (1 - a A )f(S - A) 

—A(1 - a a) log (1 - a a) > -(1 ~ ol a ~ a B A)f{S - B) 
-A(l - a A ~ Cuba) log (1 - a A - cxba) 
aBA{~f{S - B) - A log (1 - ola ~ cxba)}+ 
ot x {~f(S -A - {x}) - A log (1 - a A ~ oiba)}+ 
a A {~f(S -A- {x}) + f(S -A)- f(S - B)- 
A log (1 - a A ~ oi x ) + A log (1 - a A ) 

-A log (1 - a a - o-ba)} + 1 {f(S - A - {a?}) - f(S - A)+ 
f{S - B) + A log (1 - a a - a x ) — A log (1 - a A ) 

+A log (1 - a A - Cuba )} > 0 

By adding and removing three factors of f(S — B — {a;}), 

A log (1 — ola — ot x ) and A log (1 — ola — olba) to each of 
the factorizations above and using the definitions of g and q 
functions as previously, we can simplify the above inequality 
in the following format: 

a B A{~g(S - B) + A log (1 - a a - a x )}+ 

&x{-g(S -A- {x}) + A log (1 - a A - q B a)}+ 
a A {q{S - A) + A log (1 - a A )}+ 

+1 {-q(S - A)} - Alog (1 - a A )}+ 

{A log (1 - a A - u x ) + A log (1 - a a - a B A)} 

{—olba — a x — a A + 1} > 0 

Using the fact that a ba + a x + a A = 1 and merging the 3rd 
and 4th factorizations together, we will have: 

a B A{~g(S - B) + A log (1 - a a - a x )}+ 

OL x {-g{S - A - {x}) + A log (1 - a A - q B a)}+ 

(1 - a A ){~q{S - A) - Alog (1 - a A )} > 0 


which would hold true as long as (1) g{C) < 0 for all sets 
C and (2) \g(C)\ > A?nax(log ^-) and (3) q{C) < 0 for all 
sets C. It turns out that the third condition will be required in 
all other instances of the set B as well meaning this specific 
case does not impose any severe condition changes. However, 
it does demand a different lower bound for the absolute value 
of the first order difference of function / over sets. 

The second factorization results in two inequalities: 
(1) g(C) < 0 for all sets C and (2) \g(C)\ > 
A?7iax(log (1 + ))• To find the maximum of such 

a limit, we once again need to impose the one item with 
highest probability to {x} and assume that ola = 0 and a B 
is maximal while still less than 1 — a x . This limit is once 
again imposed so that the denominator is not equal to 0. 


Xa A log ( 
+A log ( 


q(S — H)[—1 + a A ] + 

1 — OLA 1 -OLA- OLE A - cx a 


1 - OLA - OL x 

1 - OLA 


1 — OLA — OLB A 
1 Ola Ol x 


1 — OLA~ OLB A 1 -OLA — OLB A ~ OL a 


)>0 


Depending on whether the sum of logarithmic arguments on 
the LHS is positive or negative, we can write two inequalities: 


1 — OLA 1 — OLA — OLB A — OL x 


(1) Xa A log (-- 

l-OLA- 

T A log (-— 1 ——--- 

1 — a a — olb a 1 — aA~ aBA — ol, 


ol x 1 — aA~ as a 

1 a a a x 


)>0 


q(S — A)[—l + a A ]+ 

1 - a A 1 OLA OLE A - Ol x 

Xa A log(-- 

1 — a a—' 


+Alog (- 


ola — a x 
1 ~ola 


1 — OLA — OLB A 
1 a a ol x 


-)> 


' 1 — OLA — OL B A 1 OLA OLB A — OL x 

q(S - A )[-1 + a A ] + Xa A log (-—-——-) > 0 

f — a a — a B a 


where we have simply summed the two logarithmic argu¬ 
ments. It then turns out that the logarithmic argument is 
always positive seeing as how the nominator of the fraction 
inside it is greater than the denominator. It thus follows that 
we merely need to impose that q(C) < 0 for all sets C to 
help above inequality hold true. A second scenario dictates 
when: 


,,, , , , 1 — OLA 1 ~ OLA - aBA - OL x 

(1) XaA log (-- - -) 

1 - OLA - Ol x l- aA - aBA 

\ 1 ( 1 - a A l-a A -a x \ _ 

+Alog(----) <0 h 

1 — OLA ~ OLB A 1 — OLA ~ OLB A — OL x 

q(S — A)[— 1 + a4 + 

1 - OLA 1 -OLA- OLB A - Ol x . 


A a A log ( 
+Alog (- 


1 - OLA - OL x 
1 - OLA 


1 ola olb a 

1 a a ol x 


A 1 -)> 

1 — OLA — OLB A 1 — OLA — OLBA - OL x 

q{S - A)[-l + a A ] + Alog(-— 1 ——-) > 0 

1 — OLA — OLB A 


where we have once again simply summed the two loga¬ 
rithmic arguments. It again turns out that the logarithmic 
argument is always positive seeing as how the nominator 
of the fraction inside it is greater than the denominator. 
It once more thus follows that we merely need to impose 
that q(C) < 0 for all sets C to help above inequality 
hold true. Thus 3 sufficient conditions for submodularity 
of F‘ 2 ( Sj). j = 1 are developed: (1) g(C) < 0 (2) 

q(C) < 0 (3) \g(C)\ > Alog [mox(l + ^A+ )]■ 

Thus, the total set of sufficient conditions for submod¬ 
ularity of a general M-to-N binning problem with utility 
functions as described previously is developed as follows: 

(Dff(C)<0 

(2) q(C) < 0 



(3) \g(C)\ > A log (1 + max(^, , ^)) = 

Alog(^) 

for all sets C. 

□ 


C. Lemma III. 3 

Proof. For the specific case N = 2: 

F(A) = a A f{A) + (1 - a A )f(S - A) + a A log {a A ) 
+(1 - cx A ) log (1 - a A ) 

In such a case, diminishing return property requires that 

(a A + a x )f(A + {x}) + (1 -ol a ~ a x )f(S - A - {x}) 
+(a A + a x ) log ( a A + a x ) 

+(1 -a A - a x ) log (1 -a A - a x ) - ( a A )f(A ) 

-(1 - a A )f(S -A) - (a A ) log (a A ) 
-(l-aA)log(l-ct^) > 

(a A + a BA + a x )f(B + {x}) 

+(1 - a A - a BA - ot x )f(S - B - {x}) 

+{a A + a BA + a x ) log (a A + a BA + a x ) 

+(1 - a A - a BA - a x ) log (1 - a BA - a A - a x ) 

-( a B )f(B) - (1 - a A - a BA )f{S - B) 

-(a A + a BA ) log (a A + a BA ) 

-(1 -a A - a BA ) log (1 - a A - a BA ) 


We then factorize this inequality in the following manner: 


+ {x}) + f(B) + f(S — B — {x}) - f(S - B) 


+Alog(- 


a A + a BA 1 — a A — a BA — a a 


)}+ 


" a A + a BA + a x 1 — a A — a BA 
a x {f(A + {x}) - f(B + {x}) — f(S - A - {x}) 
+f(S-B-{x})+ 
ol A + a x 1 - a A - a x - a BA , 


A log (- 


-)} 


a A + ol x + a BA 1 — a A — a x 
+a A {f(A + {x}) - f(B + {x}) - f(S-A- {x}) 
+f(S -B- {x}) - f(A) + f(B) + f(S -A)- f(S - B) 


+ log( 


a A + a x 


l — a A — a x — a BA a A + a BA 


OL A + OL x + cx BA 1 — OL A — a x 
1 - OLA 


OL A 


1 —OL A — OL BA 

+{f(S -A- {x}) - f(S-B- {x}) - f(S - A) 

, r/o , M / 1 - a a ~ a x 1 -cst A -a BA 

+f(S-B) + Alog(-- 

1 — a a — ol x — 


ot BA 


1 -Oi A 


)} 


> 0 


We now introduce two new set functions g(C + D) = 
f{C + D)- /(C) and q{C + D)= g(C + D)~ g(C) where 
C, D represent two arbitrary sets belonging to the universal 


set. We then use these new definitions to simplify the above 
inequality: 


+A log ( 


a BA {-g{B + {x}) - g(S - B) 
a A + a BA 1 — a a — a BA - a x 
a A + ol ba + ol x 1 — a a — a BA 

a x {~g(B + {x}) - g{S - A - {x})+ 


A log ( 


a a + oc x 


1 - a A — a x — a BA 


a a + ol x + a BA 


1 CtA 


)} + 


)} 


Aqza {—+ {a’}) + q(S — A^-\- 


log(- 


OLA 


1 — ola — ol x — a BA a a + a BA 


a A + ol x + OL BA 1 — OLA — Ol x 
1 - OLA 

1 — OLA — OL BA 

+{-q(S ~ A) + Alog( 


OLA 


1 — OLA — OL BA 


1 — OLA - OL x - a BA 

> 0 


1 OL A 


We aim to find sufficient conditions so that the above inequal¬ 
ity can hold true. The first factorization is the coefficients of 
ol ba > 0. We thus need to ascertain that the coefficient is 
also positive. To do so, we impose that 


-2 5 (C) + Alog( 


OLA + Ot B A 1 — OLA — OL B A — OL x 


OLA + OL BA + OL x 

> 0 


1 — OLA ~ OL BA 


)} 


for all possible sets C. We thus need to impose two sufficient 
conditions: 


(1) g{C) < 0,VC 
(2) 2|g(C)| > 


max(|A log ( 


a a + ol BA 1 ~ at a — Qba — 
OL A + ol ba + ol x 1 — a a — ol ba 


)}|) 


The second factorization is the coefficients of a x > 0. 
We once again need to ascertain that the coefficient is also 
positive. To do so, we impose that 


(1) g{C) < 0,VC 
(2) 2\g(C)\ > 


max( | A log (- 


ola 


1 ola ol x a BA 


OLA 




&BA 


1 — CLA — at x 


)l) 


We could see that the RHS of the secondary condition could 
be written as: 


Amax(log ((1 H--—-—)(1 + 

a a + ol x 


ol B a 

1 - OLA - OL x 


))) = K 

( 10 ) 


It is clear to see that the two fractions could not be maximal 
at the same time seeing as how (1) the maximal occurs 
when denominator of each is close to zero and (2) their 
denominators have opposing behavior. It thus turns out that 
the RHS as indicated in Eq. (10) is limited by: 

K < A log (max( 1 H-———)max(l -|- —— -)) 

a a + ol x 1 - ola- ol x 



The first fraction is maximized when a a = 0, a x = 7 tm and 
chba = 1 — 7 tm while the second fraction can never be as 
high. It thus follows that: 


1 



